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ABSTRACT 

Effect sizes have been increasingly emphasized in research, as more 
researchers have recognized: (a) that all parametric analyses (t- 
tests, ANOVA, etc.) are correlational, (b) that effect sizes have 
played an important role in meta-analytic work, and (c) that 
statistical significance testing is limited in its capacity to 
inform scientific inquiry. But effect sizes tend to be biased by 
sampling and measurement error. The paper illustrates and reviews 
the performance of the Wherry and the Herzberg statistical 
corrections for sampling error bias, and compares the corrections 
'•'jtb empirical estimates of sampling erro^ derived '.^sipcr 
"bootstrap" methods. A data set involving responses of 31 subjects 
on the Finding Embedded Figures Test and the Group Embedded Figures 
Test is used for illustrative purposes, to make the discussion 
concrete. It is suggested that "bootstrap" methods provide the 
researcher with important insights, and are today readily 
accessible to researchers thanks to the availability of user- 
friendly computer programs that automate the procedure, some of 
v/hich have been written for popular microcomputerr.. 



Researchers have increasingly emphasized the examination of 
effect sizes as a focal part of interpret \ng empirical results. 
Many effect size estimates (e.g., Hays, 1981; Tatsuoka, 1973) are 
available for researchers who wish to garner some insight regarding 
result importance . The simplest effect sizes are analogous to the 
coefficient of determination (r^) . For example, in analysis of 
variance the sum of squares (SOS) for an effect can be divided by 
the SOS total to compute the correlation ratio (also called eta 
squared) , just as the SOS explained in regression divided by the 
SOS total is the squared multiple correlation coefficient. Such 
statistics inform the researcher regarding what proportion o^ 
variance in the dependent variable (s) is explained by a given 
predictor. The simplest effect sizes are based on the data in hand 
and sample size is not considered as part of the calculations. 

Three factors have led to the increased emphasis on effect 
size interpretation. First, researchers have incre asingly 
recognized that all parametric analytic methods (t-test s, ANOVA. 
ANCOVA. MANOVA. Qtc.) are correlational , i.e., are special cases 
of canonical correlation analysis (Knapp, 1978; Thompson, 1984). 
Thus, canonical correlation analysis can be used to implement all 
parametric analyses, as Thompson (1988a) illustrates, just as all 
univariate parametric methods can be implemented as regression 
analyses (Cohen, 1968; Thompson, 1985). This recognition has 
stimulated researchers to realize that effect sizes analogous to 
squared correlation coefficients are just as important to interpret 
with experimental designs as squared correlation coefficients are 
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to Interpret with correlational designs. 

Second, researchers have increasingly r^gQqpi??<^ tllfi 

legitimacy and the utility of meta-analytic methods . Though 
popularized by Glass and his colleagues (Glass, 1976; Glass, McGaw 
i Smith, 1981) , meta-analytic methods actually date back to work 
by Fisher (1932), by Cochran (1937, 1943), and especially to work 
by Rosenthal (1963, 1984). Kulik and Kulik (in press) provide an 
excellent review of this history. One effect of popularized meta- 
analytic methods has been the popularization of effect size 
computations like those derived in meta-analysis. 

Third, researchers have increasingly recognized tha t 

statistical significance testing is extremely limited in its 

capacity to inform scientific inouirv (Carver, 1978; Chow, 1988; 
Huberty, 1987; Kupfersmid, 1988; Rosnow & Rosenthal, 1989; 
Thompson, 1988c, 1989a, 1989b) • Even some widely respected authors 
of prominent textbooks are sometimes not cjuite sure what role 
significance tests should play in analysis (Thompson, 1987a, 
1988e), and some dissertation authors too may be disproportionately 
susceptible to excessive awe for significance tests (Eason & 
Daniel, 1989; Thompson, 1988b) . Researchers who have had the 
fortunate experience of working with large samples (cf. Kaiser, 
1976) soon realize that virtually axl null hypotheses will be 
rejected, since **the null hypothesis of no difference is almost 
never exactly true in the population" (Thompson, 1987b, p. 14). As 
Heehl (1978, p. 822) notes, **As I believe is generally recognized 
by statisticians today and by thoughtful social scientists, the 
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null hypothesis, taken literally, Is always false.** Thus Hays 
(1981, p. 293) argues that **virtually any study can be made to show 
significant results if one uses enough subjects." The recognition 
that statistical significance testing is largely a test of sample 
size, which size the researcher already knew prior to conducting 
a significance test, has led to an increased emphasis on effect 
size interpretation. 

Because the simpler effect sizes (e.g., eta squared) 
capitalize on sampling error as part of their inherent least- 
squares or correlational logic, the simpler effect sizes do 
overestimate both the effect size in the full population and the 
effect size likely to be realized in future studies. But correction 
formulas (Maxwell, Camp & Arvey, 1981; Rosnow & Rosenthal, 1988) 
can be applied to estimate population effect sizes based on sample 
results (e.g.. Wherry, 1931), or to estimate the effect size 
estimates likely in future samples (Herzberg, 1969). Correction 
formulas are also available to adjust for attenuation due to 
measurement features such as limited reliability of measurement 
(Guilford, 1954, p. 400) or restricted variability of measurement 
(Borg fc Gall, 1989, pp. 598-599). 

The purpose of the present paper is to review two effect size 
estimates (Herzberg, 1969; Wherry, 1931) that can be computed to 
adjust for sample size influences (called "shrinkage" corrections, 
since the corrected effect size estimates tend to be smaller in 
size than uncorrected estimates) , and to compare these two 
theoretically derived statistical corrections requiring assumptions 



about the fon of sampling error Influences with methods that make 
fever assumptions and instead ground estimates in a more thorough 
empirical examination of the data in hand. With respect to 
estimates in the second genre, the present study will review the 
■bootstrap** logics developed by Efron and his colleagues. A small 
data set (n*31) is employed for heuristic purposes to make the 
discussion more concrete. 

Heuristic Example 

Data from 31 subjects who completed both the Finding Embedded 
Figures Test (FEFT) (Melancon i Thompson, 1987, 1989, 1990a, 1990b, 
1990c; Thompson « Melancon, 1990) and the widely known Group 
Embedded Figures Test (GEFT) are employed here for illustrative 
purposes. These data have not been previously reported. 

The 31 (18 females; 58.15%) subjects were undergraduate 
college students. The mean age of the subjects was 20.7 (SQM.7). 
The means for number of correct answers on the four variables of 
interest in the present example were: (a) number of right answers 
on the 35 items in FEFT Part A, 27.9 (SE=4.9); (b) number of right 
answers on the 35 items in FEFT Part B, 26.8 (SD=4.7); (c) numbsr 
of right answers on the full 70 items of the FEFT, 54.7 (SD=9,0); 
and (d) number of right answers on the 18 items of the GEFT, 11.4 
(SD=5.1) • 

The six unique correlation coefficients ( (v*(v-l) ) /2 = 
(4*3) /2) among the four variables were of interest from a 
measurement point of view. The correlation between FEFT total 
scores and GEFT total scores was of particular interest, since the 
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result i« a concurrent validity coefficient. The calculated 
correlation natrix is presented in Table 1. 



INSERT TABLE 1 ABOUT HERE. 

Corrections for Shrinkage in Estimating 
Populstion Effect Size Using Sample Results 

Various correction formulas are available to adjust for 
expected "shrinkage* when estimating population effect size using 
sample results (e.g., olkin fc Pratt, 1958). However, Carter (1979) 
notes that the various corrections tend to yield very similar 
results, especially when sample sizes are greater than 50. The 
Wherry (1931) correction formula is probably the most widely used, 
e.g., this is the correction SPSS-X uses to compute the "adjusted" 
squareU multiple correlation coefficient. Given y predictor 
variables (y-1 in the bivariate case involving a single predictor) 
and n subjejts, the Wherry correction can be expressed as: 

R^ - ((1 - R^) * (V / (n - V - 1))), 
or equivalent ly as: 

1 - ((n - 1) / (n - V - 1)) * (1 - R^) . 

Thus, for the correlation (£=.5120) between FEFT total scores 

and GEFT total scores for the 31 subjects, the correction would be: 

r **2 - ((1 - r **2) * ( V / ( n - V -1))) 
.5120 **2 - ((1 - .5120 **2) * ( 1 / (31 - 1 -1))) 
.262144 - ((1 - .262144 )*(!/( 29 ))) 
.262144 - (( .737856 ) * ( .034482 )) 
.262144 - ( 0.025443 ) 

.236700 

Table 2 presents all six unique Livariate correlation coefficients 
for the 31 subjects, after adjustment for "shrinkage" using the 
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Nherry algorltha. 



INSERT TABLE 2 ABOUT HERE. 

Correctlo na for Shrinkage in Estimating 
aanlitiAtiion Effect Siza Expected in Future Studies. 
Baaed on Sample Results 

Stevens (1986, pp. 78-84) incisively implies that researchers 
usually ground their work in empirical findings from previous 
samples . and in actual practice usually want their work to 
generalize to future samples in future research rather than to the 
unJcnowable population. Herzberg (1969) provides a correction for 
this estimate that also might be used in creating coefficient 
aggregates to evaluate variable Importance: 

1 - ((n-l)/(n-v-l))((n-2)/(n-v-2))((n+l)/n)(l-R2). 

Thus, for the correlation (£=.5120) between FEFT total scores 
and GEFT total scores for the 31 subjects, the correction would be: 



- (( n-l)/( n-v-l))*(( n-2)/( n-v-2))*(( n+1)/ n)*(l- r **2) 

- ((31-l)/(31-l-l))*((31-2)/(31-l-2))*((31+l)/31)*(l- .5120**2) 

- (( 30 )/( 29 ))*(( 29 )/( 28 ))*(( 32 )/31)*(l- .262144 ) 

- ( 1.034482 )*( 1.035714 )*( 1.032258)*( .737856 ) 

- ( 1.071428 )*( 1.032253)*( .737856 ) 

( 1.105990 * .737856 ) 

.816061 
. 183938 

Table 3 presents all six unique bivariate correlation coefficients 
for the 31 subjects, after adjustment for "shrinkage" using the 
Herzberg algorithm. 



INSERT TABLE 3 ABOUT HERE. 



^"BQQt^traP" Ennlrical Eatimatea of Effect Size Stability 

A third strategy, like the Herzberg correction, emphasizes 
interpretation based on estimated likelihood that results will 
replicate. This emphasis is compatible with the basic purpose of 
science: isolating conclusions that replicate under stated 
conditions. Notwithstanding common misconceptions to the contrary, 
significance tests do not evaluate the probability that results 
will generalize (Carver, 1978; Thompson, 1987b). 

The "bootstrap** methods developed by Efron and his colleagues 
(cf. Diaconis & Efron, 1983; Efron, 1979; Lunneborg, 1987, in 
press) are extremely powerful. Most conventional statistical 
estimates invoke the concept of the standard error (S£) of the 
statistic of interest, i.e., the standard deviation of the error 
of the estimates of population parameters. Typically, an assumption 
is made that standard errors are randomly and normally distributed 
during the sampling process, and the Sfi is derived statistically 
rather than empirically. 

For example, for flU bivariate correlation coefficients (r) 
expressed after the Fisher r-to-i transformation and involving a 
sample size of 31, the standard error (Glass & Hopkins, 1984, p. 
305) is taken to be 

1 / ( n - 3) ** .5. 
For the correlation (£=.5120) between FEET total scores and GEFT 
total scores for the 31 suojects, the correlation expressed as £ 
would be: 
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|Z| - -5 In ((1 + I r I) / (1 - I r |)) 
0.5 In ((1 + 0.512 ) / (1 - 0.512 )) 
0.5 In (( 1.512 ) / ( 0.488 )) 

0.5 In ( 3.09836 ) 
0.5 1.13087 
0.56S43 

The correlation (i:«.9418) between FEFT total scores and scores on 
the 35 item Part A of the FEFT is expressed as i as 1.75374. 

The researcher using classical statistical procedures will 
presume that the standard error of l^^^O. 56543 and the standard error 
of 1-1.75374 are exactly equal (Glass & Hopkins, 1984, p. 306), 
i.e. , .188982 (l/(n-3)**.5 « 1/28**. 5 = .035714**. 5) . This 
presumption means that the researcher will assume that the 95% 
confidence intervals about both Is are also exactly equal in their 
width, i.e., 

Z ^ and - 1.96 * .188982, or 

i + and - .3704047. 
It seems illogical to make strong assumptions that standard 
errors are randomly and normally distributed, when one has data in 
hand that can be employed to empirically estimate standard error. 
"Bootstrap** methods (Efron, 1982, 1986) provide sophisticated 
estimates of the standard errors of results, informed by the data 
in hand rather than by paltry assumptions about the likely 
distribution of sample-estimates of parameters. Thus, Lunneborg 
(1987, p. 38, his emphasis) characterizes these as " real ** 
estimates. And various types of confidence intervals can be 
constructed using these methods (Buckland, 1985; Efron, 1987; Efron 
& Tibshirani, 1986; Lunneborg, 1986). 
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^ Conceptually, "bootstrap" methods involve copying the data 
set over again and again nany, many times into a large "mega" data 
set. Then dozens (or hundreds or thousands) of different samples 
are drawn from the "mega" file, and results are computed separately 
for each sample and then averaged. The method is powerful because 
the analysis considers so many configurations of subjects and 
informs the researcher regarding the extent to which results 
generalize across different configurations of subjects. Lunneborg 
(1987) has offered some excellent computer programs that automate 
this logic for univariate applications; Thompson (1988d) and 
Lambert, Wildt and Durand (1990) provide similar software for some 
multivariate applications. Borrello and Thompson (1989) and Scott, 
Thompson and Sexton (1989) illustrate applications of these 
Bethods • 

Table 4 presents selected entries from bootstrap estimation 
of the correlation matrix presented in Table 1, based on 1,000 
resamplings of the data from the 31 subjects. These analyses were 
conducted on a microcomputer using Lunneborg 's (1987) program , 
CORBOOT. 

INSERT TABLE 4 ABOUT HERE. 

Lunneborg's (1987) program, BOOTLV, was run to derive various 
descriptive statistics for the results associated with the 1,000 
resamplings. Some of these results are presented in Table 5. The 
tabled results suggest some important benefits of "bootstrap" 
estimation procedures. For example, the fact that the standard 
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deviation of the estinates over 1,000 resamplings (empirical 
estimates of standard error/ of the t for the variable pair GEFT 
with FEFT Total (SD-. 12698800) did not equal the £D for the 
variable pair FEFT Part A with FEFT Total (§E=, 02987297) 
illustrates that it may not be tenable to assume that all standard 
errors for a fixed sample size are exactly equal. Furthermore, the 
fact that mean (e.g., .50920460) and median (e.g., .52025070) 
estimates of the r for the variable pair GEFT with FEFT Total were 
not exactly equal illustrates that the assumption that sampling 
error is normally distributed may also not always be exactly true. 

INSEl T TABLE 5 ABOUT "HERET 

Lunneborg's (1987) program, BOOTCI, was run on the 
■icrocomputer to calculate 95% confidence intervals. Selected 
results for the example are presented in Table 6. The program will 
compute any width intervals^ the user desires, and also provides 
intervals constructed using several different logics. 

INSERT TABLE 6 ABOUT HERE. 

In the present example the conventional standard error for e 
expressed as £ was .188982 (l/(n-3)**.5 = 1/28**. 5 = . 035714** . 5) . 
The empirical estimates, i.e., the 5D over 1,000 resamplings, was 
SD of r - .12698800; the £ form of this S£ equals .12767. The 
smaller empirical estimate of the results in narrower 95% 
confidence intervals for the bootstrap procedure, as illustrated 
in Table 6. 

10 



ERIC 



13 



To illustrate the invariance of bootstrap estimates (at least 
when 1,000 resamplings are conducted), a new set of 1,000 
resamplings was isolated. Descriptive statistics for the analysis 
are presented in Table 7, and can be compared with the results 
reported in Table 5. 

INSERT TABLE 7 ABOUT HERE. 

Discussion 

Three general comments can be made regarding the three effect 
size corrections reported here. First, the Wherrv fl931 l and the 
Herzbera fl969^ corrections tend to be large r as e ither gffectg, 
sizes or sample sizes become smaller , as illustrated by Thompson 
(1990). Thus, with a very large effect size approaching 1.0, or a 
large sample size, or both, it will matter less which, if any, 
statistical corrections the researcher applies in estimating effect 
sizes. 

However, meta-analyses of substantive findings (effect sizes 
tend to be larger in reliability or criterion-related validity 
studies) suggest that effect sizes do not tend to get much larger 
than 25 to 33%. For example, Cohen's (1988) perusal of published 
research suggests that a correlation ratio of around 25% 
should be considered large in terms of typical findings across 
disciplines. The empirical meta-analytic work of Glass and others, 
which has yielded some additional ways of evaluating effect size, 
has also led to similar conclusions: 

In none of the dozen or so research literatures that 
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ve have Integrated In the past five years have we 
ever encountered a cross*va 1 Idated mult iple 
corr«)latlon between study findings and study 
characteristics that was larger than approximately 
0.60. That is, I haven't seen a body of literature 
in which we can account for much more than a third 
of the variability in the results of studies , [which 
is distinct from talking about results for only one 
smaller group of subjects]. (Glass, 1979, p. 13) 
Second, as suggested by the illustration, the Herz bera f 19691 
correction tends to be more conservative than t he Wherry (19311 
correction . However, this only makes sense. Since the Herzberg 
correction is used to estimate effect sizes that may be isolated 
in a new sample, the estimate in effect must correct both for 
sampling error influences for the data in hand and for the sampling 
error that will recur in new samples. The Wherry correction only 
presumes one set of sampling error influences, i.e., sampling error 
for the data in hand. 

Third, as illustrated by the case study, statistical estimates 
of standard errors for effect sizes and emp irically derived 
estimates mav differ, and the confidence intervals deri ved using 
the two approaches mav also differ. It seems illogical to be 
prepared to accept the sample-based r^stimates, sunh as an £ or a 
squared t , and to then to estimate the standard error of the 
sample-grounded estimates based on strong assumptions rather than 
the data in hand. Such illogic may have been necessary in the era 
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preceding both Mie elaboration of "bootstrap" logic (Efron, 1979, 
1982, 1986, 1987; Efron & Tibshirani, 1986; Lunneborg, 1985, 1987, 
in press; Lunneborg & Tousignant, 1985) and the widespread 
availability of computer programs that readily implement the 
necessary calculations (e.g., Lunneborg, 1987; Thompson, 1988d) . 
But a new day upon us. 

Put differently, assump^ *.ons that sampling error is normally 
distributed »ight be tenable if researchers lived in a world in 
which they routinely drew true probability samples from defined 
populations, and if in this world all randomly selected subjects 
then participated in studies. But most of us inhabit a world in 
which samples of convenience may be necessary (though thoughtful 
usually compare the characteristics of the sample of convenience 
with those of the population) , and a world in which some subjects 
do not agree to participate and still more withdraw subsequent to 
agreeing to participate. 

Bootstrap logics are appealing, because they focus on the sina 
qua non of science, i.e., replication. As Thompson (1989b, p. 4) 
notes, "significance, importance, and replicability are all 
important issues in research. [But] Too many researchers attend 
only to issues of significance in their research. And in some 
respects, statistical significance may be the least important 
element of this research triumvirate." 
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Table 1 

Blvarlate Correlation Matrix 
(n-31) 



FEFT Form A 
FEFT Form B 
FEFT Total 
6EFT Total 



(Y-35) 
(Y-35) 
(Y-70) 
(Y-18) 



FEFTA 

.7617 
.9418 
.4239 



FEFTB 



.9352 
.5402 



FEFTTOT 



.5120 



Table 2 

Effect Size Estimates Invoking the Wherry Correction 

(n»31) 

Variables Being r from 2 Adjusted 
Correlated Table 1 r r square 



FEFT B 

FEFTTOT 

FEFTTOT 

GEFT 

GEFT 

GEFT 



FEFT A 
FEFT A 
FEFT B 
FEFT A 
FEFT B 
FEFTTOT 



0.7617 
0.9418 
0.9352 
0.4239 
0.5402 
0.5120 



0.580186 
0.886987 
0.874599 
0.179691 
0.291816 
0.262144 



0.565710 
0.883090 
0.870274 
0.151404 
0.267395 
0.236700 



Table 3 

Effect Size Estimates Invoking the Herzberg Correction 

(n-31) 

Variables Being r from 2 Adjusted 
Correlated Table 1 r r square 



FEFT B 

FEFTTOT 

FEFTTOT 

GEFT 

GEFT 

GEFT 



FEFT A 
FEFT A 
FEFT B 
FEFT A 
FEFT B 
FEFTTOT 



0.7617 
0.9418 
0.9352 
0.4239 
0.5402 
0.5120 



0.580186 
0.886987 
0.874599 
0.179691 
0.291816 
0.262144 



0.535690 
0.875008 
0.861307 
0.092746 
0.216755 
0.183938 



ERIC 



22 



25 



Table 4 

Calculated Correlation Coefficients for the Sample of 31 
Subjects and Seven of 1,000 Random Resamplings of the 31 Subjects 



Sample 

0 

1 
2 
3 
4 
5 

• • • 4 

999 
1000 



Estimates of the 



42388050 
94184750 
,44996280 
,93473370 
,40545540 
,89779330 
,26309840 
,89433320 
,69394230 
.96291700 
.50877910 
.93972070 

.57282050 
.95198860 
.34975410 
.89981100 



.54021650 
.93515300 
.38785020 
.92903820 
.57462420 
.91637680 
.60893980 
.87400320 
.79731420 
.96486490 
.65551280 
.93641970 

.68778800 
.94900570 
.40709220 
.88771040 



Six Unique n's 
.51195450 .76173090 



.45017010 
.54441740 
.48483890 
.77426050 
.61955510 

.66222770 
.42246560 



.73693000 
.64 641050 
.56424870 
.85819620 
.75999140 

.80693140 
.59790740 



Note . Sample 0 Involves the results calculated for the 31 subjects. 
The correlation coefficients are presented in the order: (a) 6EFT 
X FEFT Part A; (b) GEFT x FEFT Part B; (c) GEFT x FEFT total; (d) 
FEFT Part A x FEFT Part B; (e) FEFT total x FEFT Part A; and (f) 
FEFT total x FEFT Part B. 



Table 5 

Bootstrap Results Across 1,000 Resamplings 
of 31 Subjects in Random Configurations 



Statistic 

£ for 31 Subjects 
Mean of 1,000 Samples 
SD (akin to SE) 
Median of 1,000 Samples 
Lowest Estimate 
Largest Estimate 
Range 



GEFT X 
FEFT Total 

.51195 
.50920460 
.12698800 
.52025070 
-.006179 
.8108 
.8170 



FEFT A X 
FEFT Total 

.94185 

.93533360 

.02987297 

.94288290 

.7603 

.9907 

.2305 
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Table 6 

Conventional and Four Bootstrap Confidence Intervals 
for the GEFT x FEET Total Variable Pair 

Conventional Estimates 

t 0.51195 

£ as 2 0.56536 

statistical SE 0.188982 

95% CI for Z. 0.194954 to 0.935765 



Bootstrap Estimates 



£ 

£ as 2 
Bapirioal 



SE 



95% CI for £ 
as Z. 

95% CI for £ 
as Z. 

95% CI for £ 
as Z 

95% CI for £ 
as Z 



0.51195 
0.56536 
0.12699 

Symmetric (Normal Theory) 
0.26263 to 0.76128 
0.26893 to 0.99925 
Percentile Method 
0.23514 to 0.72782 
0.23962 to 0.92407 
Bias Correctea Percentile 
0.21984 to 0.72028 
0.22348 to 0.90822 
Minimum width 
0.26930 to 0.75235 
0.27610 to 0.97834 



Table 7 

Bootstrap Results A New 1,000 Resamplings 
of 31 Subjects in Random Configurations 



Statistic 

£ for 31 Subjects 
Mftan of 1,000 Samples 
SD (akin to SE) 
Median of 1,000 Samples 
Lowest Estimate 
Largest Estimate 
Range 



GEFT X 
FEFT Total 

.51195 

.51206590 

.12820730 

.52079060 

.06310 

.8790 

.8159 



FEFT A X 
FEFT Total 

.94185 
.93687050 
02870789 
.9A280670 
.S200 
.9907 
.1706 
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APPENDIX A: 
Item Analysis for Heuristic FEFT Data 
(n-31) 



Font 


A 




iteB 




P 


A 1 




0.94 


A 2 




0.84 


A 4 




0.81 


A 5 




0.S2 


A 9 




O.SS 


All 




0.81 


A13 




0.84 


A17 




0.77 


A18 




1.00 


A23 




0.94 


A25 




0.81 


A26 




0.68 


A27 




0.94 


A28 




0.90 


A30 




0.81 


A31 




0.84 


A32 




0.87 


A3 3 




0.87 


A3 4 




0.84 


A3 5 




0.87 


A3 6 




0.6S 


A37 




0.71 


A3 8 




0.32 


A39 




0.74 


A40 




0.S2 


A 3 


LOl 


0.97 


A 6 


L02 


0.84 


A 7 


L03 


0.3S 


A 8 


TAJ 

L04 


0.71 


AlO 


LOS 


0.S2 


A12 


L06 


0.61 


A14 


L07 


0.90 


A15 


L08 


0.48 


A16 


L09 


0.87 


A19 


LIO 


0.87 


A20 


Lll 


0.77 


A21 


L12 


0.94 


A22 


L13 


0.97 


A24 


L14 


0.87 


A29 


LIS 


0.77 


Non-Linking 





Mean 0.774 
SD 0.1S7 
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Corr . r 


Corr. r 


Corr. r 




with 39 


with 34 


with 17 


SD 


FEFT A 


FEFT A 


GEFT 


0.2S 


-0.01 


0.05 


0.49 


0.37 


0.33 


0.38 


0.28 


0.40 


-0.01 


-0.08 


-0.09 


O.Sl 


0.45 


0.47 


0.01 


O.Sl 


0.43 


0.38 


0.43 


0.40 


0.25 


0.23 


0.17 


0.37 


0.27 


0.19 


0.24 


0.43 


0.27 


0.29 


0.10 


0.00 


— 


— 


— 


0.2S 


0.35 


0.38 


0.31 


0.40 


0.28 


0.27 


0.22 


0.48 


0.33 


0.31 


0.09 


0.2S 


0.18 


0.24 


0.15 


0.30 


0.05 


0.04 


-0.21 


0.40 


0.30 


0.27 


0.12 


0.3:* 


0.50 


0.49 


0.26 


0.34 


0.40 


0.35 


0.09 


0.34 


0.40 


0.37 


0.01 


0.37 


0.66 


0.61 


0.03 


0.34 


0.42 


0.35 


0.03 


0.49 


0.42 


— 


0.15 


0.46 


0.09 


— — 


-0.18 


0.48 


0.27 


— — 


-0.08 


0.44 


0.43 


— — 


0.22 


O.Sl 


0.32 


-— 


0.29 


0.18 


0.03 


0.03 


-0.02 


0.37 


0.25 


0. lo 


0.45 


0.49 


0.07 


0.06 


0.33 


0.46 


0.64 


0.62 


0.26 


O.Sl 


0.29 


0.28 


-0.04 


0.50 


0.27 


0.30 


0.18 


0.30 


0.21 


0.25 


0.24 


0.51 


0.75 


0.76 


0.45 


0.34 


0.19 


0.12 


0.09 


0.34 


0.24 


0.28 


0.01 


0.43 


0.27 


0.27 


0.18 


0.25 


0.16 


0.16 


-0.11 


0.18 


0.45 


0.50 


0.23 


0.34 


0.26 


0.30 


0.05 


0.43 


0.55 


0.57 


0.30 


0.379 


0.306 


0.294 


0.131 


0.109 


0.157 


0.160 


0.172 
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Linking 










0.174 


Mean 


0.763 


0.374 


0.307 


0.319 


SD 


0.183 


0.108 


0.196 


0.201 


0.169 


All Form A 












Mean 


0.770 


0.377 


0.299 


Oc305 


0.144 


SD 


0.167 


0.109 


0.178 


0.180 


0.171 








Corr. r 


Corr. r 


Corr . r 


Form B 






with 39 


with 34 


with 17 


Item 


P 


SD 


FEFT B 


FEFT B 


GEFT 


B 2 


0.42 


0.50 


0.40 


0.38 


0.38 


B 3 


0.26 


0.44 


0.21 


0.21 


0.22 


B 4 


0.55 


0.51 


0.15 


0.18 


0.04 


B 7 


0.81 


0.40 


0.30 


0.31 


-0.09 


B 8 


0.81 


0.40 


0.23 


0.20 


0.12 


BIO 


0.74 


0.44 


0.33 


0.34 


0.15 


B12 


0.77 


0.43 


0.43 


0.46 


0.21 


B13 


0.23 


0.43 


0.38 


0.31 


0.43 


B16 


0.74 


0.44 


0.32 


0.36 


0.19 


B19 


1.00 


0.00 


— 


— 


— 


B20 


0.94 


Z.25 


0.24 


0.31 


0.23 


B23 


0.87 


0.34 


0.20 


0.23 


0. 16 


B24 


0.81 


0.40 


0.39 


0.43 


0.18 


B25 


0.81 


0.40 


0.44 


0.43 


0.31 


B27 


1.00 


0.00 


— 


— — 




B30 


0.55 


0.51 


0.35 


0.42 


0. 16 


B31 


0.94 


0.25 


0.43 


0.46 


0. 15 


B33 


0.90 


0.30 


0.26 


0.21 


0.13 


B34 


1.00 


0.00 


— 






B35 


0.81 


0.40 


0.34 


0.33 


0.22 


B36 


0.19 


0.40 


0.21 




0.25 


B37 


0.16 


0.37 


0.11 


— 


0.23 


B38 


0.42 


0.50 


0.15 


— 


0.34 


B39 


0.42 


0.50 


0.28 


— — 


0.35 


B40 


0.52 


0.51 


0.12 


— — 


0.25 


B 1 LOl 


0.97 


0.18 


0.50 


0.48 


0.23 


B 5 L02 


0.81 


0.40 


0.40 


0.39 


0.18 


B 6 L03 


0.39 


0.50 


-0.02 


-0.02 


0. 12 


B 9 L04 


0.77 


0.43 


0.24 


0.26 


0.20 


Bll LOS 


0.68 


0.48 


0.29 


0.30 


0.23 


B14 L06 


0.74 


0.44 


0.24 


0.22 


0.29 


B15 L07 


0.90 


0.30 


0.30 


0.26 


0.24 


B17 LOB 


0.48 


0.51 


0.23 


0.22 


0.26 


B18 L09 


0.90 


0.30 


0.28 


0.33 


0.35 


B21 LIO 


0.87 


0.34 


0.48 


0.50 


0.30 


B22 Lll 


0.81 


0.40 


0.30 


0.22 


0.10 


B26 L12 


0.94 


0.25 


0.13 


0.14 


-0.06 


B28 L13 


0.94' 


0.25 


0.48 


0.46 


0.20 


B29 L14 


0.84 


0.37 


-0.01 


-0.04 


0.26 


B32 LIS 


0.84 


0.37 


0.51 


0.52 


0.35 
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Non-Linking 



Mean 


0.666 


0.365 


0.318 


0.328 


0.210 


SD 


0.266 


0.153 


0.086 


0.092 


0.112 


Linking 












Mean 


0.791 


0.368 


0.291 


0.283 


0.218 


SD 


0.160 


0.094 


0.163 


0.167 


0.101 


All Fonn 


B 










Mean 


0.713 


0.366 


0.266 


0.306 


0.197 


SD 


0.240 


0.134 


0.146 


0.134 


0.118 



ERIC 



27 



Appendix 



EKD 

. S • Dopt • of Educatiej . 

Office of Education 

Research and 
Improvement (OERl) 

ERIC 

Date Filmed 
March 29, 1991 



